Interpretation of Image Flow: A Spatio-Temporal Approach

Research on the interpretation of image flow (or optical flow) until now has mainly focused on instantaneous flow fields. This limits the scope of the problems that one can address and the accuracy of the results. Here we extend a previous formulation of the problem to incorporate temporal variation of image flow. We illustrate our approach by solving specific cases which are of practical significance including simple cases of non-rigid and non-uniform motions. The formulation is general in that it is applicable to any situation provided that the scene geometry, the scene transformation, and the image flow are all ‘‘smooth’’ or analytic. For the case of rigid and uniform motion we have obtained some results which are of practical value. We have shown that only the first-order spatial and temporal derivatives of image flow are sufficient to recover the local surface orientation and motion; second-order (or higher order) derivatives whose measurement is unreliable are not necessary. (In comparison, previous methods use up to second-order spatial derivatives.). Further, the representation and the solution method used here have some advantages in comparison with the existing approaches; they facilitate a uniform approach to all cases of rigid motion (including the case of interpreting instantaneous visual motion). Index Terms Three-dimensional interpretation of image flow, optical flow, motion analysis, surface structure and transformation recovery.